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Abstract —In this paper, an aggregate game is adopted for 
the modeling and analysis of energy consumption control in 
smart grid. Since the electricity users’ cost functions depend 
on the aggregate energy consumption, which is unknown to 
the end users, an average consensus protocol is employed to 
estimate it. By neighboring communication among the users 
about their estimations on the aggregate energy consumption, 
Nash seeking strategies are developed. Convergence properties 
are explored for the proposed Nash seeking strategies. For 
energy consumption game that may have multiple isolated Nash 
equilibria, a local convergence result is derived. The conver¬ 
gence is established by utilizing singular perturbation analysis 
and Lyapunov stability analysis. Energy consumption control 
for a network of heating, ventilation, and air conditioning 
(HVAC) systems is investigated. Based on the uniqueness of 
the Nash equilibrium, it is shown that the players’ actions 
converge to a neighborhood of the unique Nash equilibrium 
non-locally. More specially, if the unique Nash equilibrium is 
an inner Nash equilibrium, an exponential convergence result 
is obtained. Energy consumption game with stubborn players 
is studied. In this case, the actions of the rational players can 
be driven to a neighborhood of their best response strategies by 
using the proposed method. Numerical examples are presented 
to verify the effectiveness of the proposed methods. 

Index Terms —Energy consumption control; aggregate game; 
Nash equilibrium seeking 

I. INTRODUCTION 

Demand response schemes are designed to motivate the 
electricity users to adjust their energy consumptions to the 
desired profile based on supply conditions [1], Due to the 
benefits, such as improving system reliability, efficiency, 
security and customer bill savings, great efforts have been 
dedicated to control and optimization problems related to 
demand response in smart grid (e.g., see [2]-[5] and the refer¬ 
ences therein). The electricity users are categorized as price- 
takers and price-anticipating users in the existing literatures 
[7]. The price-takers schedule their energy consumptions 
regardless of their effect on the electricity price. In contrast, 
the price-anticipating users consider the impact of their 
energy consumptions on the electricity price. Industrial users 
and commercial users with large energy consumption are 
typical examples of price-anticipating users. Taking the price 
as a function of the aggregate energy consumption brings 
up coupled energy consumption control problems among the 
electricity users. 
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Game theory is an effective modeling and analysis tool to 
deal with the interactions among the electricity consumers 
[2]. Game theoretical approaches such as stackelberg game, 
evolutionary game, differential game, just to name a few, 
have been extensively utilized to design and analyze demand 
response schemes in smart grid (e.g., [8]-[21]). For exam¬ 
ple, in [8], Stackelberg game was leveraged to model the 
interaction between the demand response aggregators and 
generators. The Nash equilibrium of the game among the 
generators was computed via solving a centralized quadratic 
optimization problem. In [9], the authors considered peak- 
to-average ratio (PAR) minimization and energy cost mini¬ 
mization by designing a non-cooperative energy consumption 
game. By communicating among the players on their sched¬ 
uled daily energy consumptions, a Nash seeking strategy 
was proposed. A coupled-constraint game was used in [10] 
for scheduling residential energy consumption. Noticing that 
the best response algorithm suffers from privacy concern, a 
gradient projection method based on estimations of the price 
changing trend was proposed at the cost of computation time 
and memory. Multiple utility companies were considered in 
[11] where the authors used a two-level game approach to 
manage the interactions. The competition among the utility 
companies was described by a non-cooperative game. The 
users were modeled as evolutionary game players. Consid¬ 
ering the system dynamics, a differential game approach 
was proposed in [12]. In this paper, the energy consumption 
control is considered as an aggregate game [22]-[24] on 
graph. 

Related Work: The main objective of this paper is to 
achieve Nash equilibrium seeking (e.g., see [26]-[30] and 
the references therein for an incomplete reference list) in 
energy consumption game. Different from many Nash seek¬ 
ing strategies where full communication among the players 
is used, in [25] and [30], the Nash equilibrium is attained 
by utilizing neighboring communication. An aggregate game 
(e.g., see [22]-[25]) in which the players interact through 
the sum of the players’ actions was investigated in [25]. 
By utilizing a gossip algorithm, the players can search for 
the Nash equilibrium through neighboring communication. 
This idea was further generalized in [30]. The players were 
considered to be generally interacting with each other. The 
game on graph was then solved via using a gossip algorithm. 
Similar to [25] and [30], this paper leverages the idea of game 
on graph to solve energy consumption game in smart grid. 

This paper aims to solve energy consumption control for 
a network of price-anticipating electricity users. Compared 
with the existing works, the main contributions of the paper 


are summarized as follows. 

« An aggregate game is adopted for the modeling and 
analysis of energy consumption control in smart grid. 
By using game on graph, the users update their ac¬ 
tions based on the communication with their neighbors. 
This scheme reduces the communication between the 
electricity users with the centralized agent (e.g., energy 
provider). Hence, the single-node congestion problem 
is relieved. 

• An aggregate game that may admit multiple isolated 
Nash equilibria is firstly considered. Based on an av¬ 
erage consensus protocol, a Nash seeking strategy is 
proposed for the players. An energy consumption game 
for a network of heating, ventilation and air condition¬ 
ing (HVAC) systems, in which the Nash equilibrium 
is unique, is investigated. The Nash seeking strategy 
is designed based on a primal-dual dynamics. More 
specifically, if the unique Nash equilibrium is an inner 
Nash equilibrium, an exponential convergence result 
is derived. Energy consumption game with stubborn 
players is studied. It is shown that with the presence of 
stubborn players, the rational players’ actions converge 
to a neighborhood of their best response strategies. The 
proposed Nash seeking strategies serve as alternative 
approaches for the gossip-algorithm in [25] to solve 
aggregative game on graph. 

« The end-users only need to communicate with their 
neighbors on their estimations of the aggregate energy 
consumption. They don’t need to share their own energy 
consumptions with their opponents. Hence, the privacy 
of the electricity users is protected. 

The rest of the paper is structured as follows. In Section 
El some related preliminaries are provided. System model 
and the problem formulation are given in Section [III] A 
general energy consumption game that may have multiple 
Nash equilibria is studied in Section Hvl without considering 
the constraints. Energy consumption game among a network 
of HVAC systems is investigated in Section[V] In Section IvTl 
numerical examples are provided to verify the effectiveness 
of the proposed methods. Conclusions and future directions 
are stated in Section IVHl 


II. Preliminaries and Notations 


In this paper, R represents for the set of real numbers, R + 
stands for the set of non-negative real numbers and R++ is 
the set of positive real numbers. Furthermore, diag{£;j} for 
i £ {1,2, ■ • • ,N} is defined as 


diagjfcj 


h 0 

0 k 2 


0 • • • fcjv 


and [hi] vec for i £ {1,2,--- , iV} is defined as [hi] vec = 
[hi, h 2 , * * * , h N ] T . 


A. Game Theory 

Below are some definitions on game theory [24]-[34]. 

Definition 1: A game in normal form is defined as a triple 
T = {N,X, C} where N = {1,2, ,N} is the set of N 

players, X = X\ x • • • x Xn, Xi C R is the set of actions 
for player i, and C = (C\, • ■ ■ , Cn) where C, is the cost 
function of player i. 

Definition 2: (Potential game) A game T is a potential 
game if there exists a function P such that 

dh dh ’ c ’ 

V* G IN. Furthermore, the function P is the potential 
function. In (|T}. I, denotes the action of player i and, 1_, 
denotes all the players’ actions other than the action of player 
i, i.e., — [^i, , li— l, ; In] 

Definition 3: (Aggregate game) An aggregate game is a 
normal form game with the players’ cost functions depending 
only on its own action and a linear aggregate of the full action 
profile. 

Definition 4: (Nash equilibrium) Nash equilibrium is an 
action profile on which no player can reduce its cost by 
unilaterally changing its own action, i.e., an action profile 
1* = G X is the Nash equilibrium if for all the 

players 

a(/?,l* i )<C'i(ii,l* i ),VfG]N (2) 

for all li £ Xi. 

For an aggregate game T with the aggregate function being 
I = JJtLi h, an equilibrium of the aggregate game T is an 
action profile 1* G X on which Mi £ IN 

N N 

J2 l *j)< c Si,k+ Y l*),Vli£Xi 
j=i ijy* 

and the associated equilibrium aggregate is denoted as l* = 
Z^i= 1 H ■ 

B. Graph Theory 

For a graph defined as G = ( V , E) where E is the 
edge set satisfying E C V x V with V = {1,2, ••• ,N} 
being the set of nodes in the network, it is undirected if 
for every (i. j) £ E, ( j,i ) G E. An undirected graph 
is connected if there exists a path between any pair of 
distinct vertices. The element in the adjacency matrix A 
is defined as = 1 if node j is connected with node i, 
else, <iij = 0. The neighboring set of agent i is defined as 
J\fj = {j £ V\(j, i) £ E}. The Laplacian matrix for the 
graph L is defined as L = D — A where D is defined as a 
diagonal matrix whose ith diagonal element is equal to the 
out degree of node i, represented by a ij- 

C. Dynamic Average Consensus 

Theorem 1: [39], [40] Let G be a connected, undirected 
graph, L be the Laplacian of G. Then, for any constant ui £ 
R n , the state of the following system 
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Fig. 1: A simplified illustration of the electricity buying and 
selling model. 


with arbitrary initial conditions x(0),y(0) £ R N remains 
bounded and x(t) converges exponentially to ^l T u;l as 
t —> oo where 1 denotes an N dimensional column vector 
composed of 1. 

D. Saddle Point 

The pair (x*,y*) is the saddle point of F(x, y) if 

F(x*,y) <F(x*,y*) <F(x,y*), (3) 

is satisfied [5], [6]. 

III. System Model and Problem Formulation 

Consider an electricity market with a network of N 
price-anticipating users. The simplified illustration of the 
electricity buying and selling model is shown in Fig. Q] 

The users are equipped with an energy-management 
controller (EMC) and an advanced metering infrastructure 
(AMI) [38]. The EMC is used to schedule the electricity 
usage for the corresponding user. The AMI enables bidirec¬ 
tional communication among the electricity users and the 
centralized agent (e.g., the energy provider). The communi¬ 
cation between the electricity users and their neighbors can 
be modeled by an undirected and connected graph. In this 
paper, we suppose that the electricity users schedule their 
energy consumption by minimizing their own costs. 

Let li be the energy consumption of user i. Then, the cost 
of user i can be quantified as [38], 

C i (l i J) = V i (l i ) + P(I)l i , (4) 

where l is the aggregate energy consumption of all the 
electricity users, i.e., I = Furthermore, Vi(U ) is the 

load curtailment cost [38], The term P{l)U represents the 
billing payment for the consumption of energy li where the 
price P(l) is a function of the aggregate energy consumption 

I. 

For each user, the energy consumption should be within 
its acceptable range, i.e., U £ [Z™ m , Zf 18 *] where Z™ ln < Z™ ax 


and Z™ m , Z™ ax are the minimal and maximal acceptable 
energy consumption for user i, respectively. The problem 
is defined as follows. 

Problem 1: (Strategy Design for the Electricity Con¬ 
sumers in the Energy Consumption Game) In the energy 
consumption game, the electricity users are the players. The 
energy consumption li and Ci(li,l) are the action and cost 
of player i, respectively. Player i’s objective is defined as 

min Ci(liJ) 

subject to lf n <k< Z“ ax , i £ IN, 

where IN = {1, 2, • • - , N} denotes the set of electricity 
users. The aggregate energy consumption l is unknown to 
the players. Suppose that the pure-strategy Nash equilibrium 
exists and is isolated. Furthermore, Ci,i £ IN are smooth 
functions over R N . Design a control strategy for the players 
such that they can search for the Nash equilibrium. 

Remark 1: In practice, providing the aggregate energy 
consumption I to all the users is challenging for the cen¬ 
tralized agent when the users are dynamically updating their 
actions. Hence, we consider the Nash equilibrium seeking 
under the condition that the users have no access to the 
aggregate energy consumption l. But the users are allowed 
to communicate with their neighbors on the estimations of 
the aggregate energy consumption. Furthermore, we suppose 
that the centralized agent can broadcast the total number of 
electricity users N to all the electricity users in the network. 

In summary, the energy consumption control considered 
in this paper is based on the following assumption. 

Assumption 1: The electricity users can communicate 
with their neighbors via an undirected and connected graph. 
Furthermore, the total number of the electricity users, N, is 
known to all the electricity users. 

IV. Energy Consumption Game Design and 
Analysis 

In this section, the energy consumption game is considered 
in the general form (the pricing function is not specified). For 
simplicity, the constraints Z“ ln < U < Vf ayi ,i £ IN, are not 
considered in this section. 


A. Game Analysis 

Before we proceed to facilitate the subsequent analysis, 
the following assumptions are made. 

Assumption 2: [29] There exists isolated, stable Nash 
equilibrium on which 




d 2 c t 

dP 


(1*) > 0,V* G IN 


( 5 ) 


where 1* denotes the Nash equilibrium. 






























Assumption 3: [29] The matrix 

~ 3 ^a*) fea*) 
^(i*) ^fra*) 


B = 


d 2 C i 

dhdlN 


(i*) 


9 £ 9 Z 2 


9 2 Cjv 

dl\dln 


(V) 


9 2 C« 

""Bp 

Ui N 


( 1 *) 


( 6 ) 


is strictly diagonally dominant. 

Inspired by [25], let denote player i’s estimation on the 
aggregate energy consumption. By using the estimations, the 
players’ objectives can be rewritten as: 

Problem 2: Player i’s objective is 


min Ci(li, Df), 


(7) 


where 79, = Jf,f= i lj,Vi G IN. 

In the following, a consensus based method will be 
proposed to search for the Nash equilibrium of the energy 
consumption game (without considering the constraints). 

B. Nash Equilibrium Seeking for the Aggregate Energy Con¬ 
sumption Game 

Based on the consensus protocol in [39]-[40], the Nash 
seeking strategy for player i,i £ IN is designed as 


je Ni 


je-Ni 


ki — (Di Dj) 

J'6 Ni 


h = + P(Di) + 

oh ODi 


(8b) 

(8c) 


where A h denotes the neighboring set of player i, ki = 
Ski,i £ IN, 6 is a small positive parameter and ki is a fixed 
positive parameter. Furthermore, Ki,i £ IN are intermediate 
variables. 

Writing © in the concatenated form gives 


D 


k 



1 = -<*([ 


-I-L —1/ 

L 0 

dV x 


D 

n 


Nl 

0 


dh 


P ( D i) + k^§^-]vec), 


(9a) 

(9b) 


where k is defined as k = diag{fci}, i £ IN, D, rz, 1 
are the concatenated vectors of I);, Ki, /,, respectively. Let 
U = [U, p] be an N x N dimensional orthonormal matrix 
such that p T L = 0 where p is an N dimensional column 

k 


vector. Furthermore, let k = U 

dimensional column vector [5]. 
can be rewritten as 


-I-L -LU 
U t L 0 


where k is an N — 1 


'hen, the closed-loop system 
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k 



D 
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+ 


N 1 
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* ~ dk([ dll + + k dDi wc) ’ 


(10a) 

(10b) 


and 


kfj, — 0, 


( 11 ) 


which indicates that n^ft) = /c(0). 

Suppose that D e (l), /$ e (l) are the quasi-steady states of 
D and k, respectively, i.e.. 


-I-L -LU 

" D e (l) ' 


' ATI ' 

U t L 0 

[ ^(1) J 

+ 

0 


= 0, 


( 12 ) 


for fixed 1. Note that D e (l), k, <: ( 1) is unique for fixed 1 as 
—I-L —LU 
U T L 0 

direct calculation, it can be derived that 


the matrix 


is Hurwitz [40]. Then, by 


Di = -Di - ^ (A - Dj) - ^ - K j) + N h (8a) 


1 ~~i 

D e (l) 


' ATI ' 

L 0 

[ ui(k e (i)r,K,] T \ 

+ 

0 


= 0 , 

(13) 

which indicates that D e (l) is the equilibrium of the original 
system. Hence, by using Theorem[U it can be concluded that 
D e (l) = 1 'ffiLi k f° r fixed 1- 

Theorem 2: Suppose that Assumptions [l© hold. Then, 
there exists a positive constant 5* such that for every 
0 < S < 5*, (1(f), D(f ),k(t)) converges exponentially to 
(l*i 1 Sill J*) ^ 6 (1*)) under ([IS given that ||1(0) - 1*||, 
||D(0) — 1 J2iLi ^11’ ||^(0) — K e (l*)|| are sufficiently small. 
Proof: Let r = St. Then, in the r-time scale, the reduced- 
system is 


r s dI3 i 


6 dz 


sdK 


dr - 



-I-L -LU 
U t L 0 



D 


' Nl ' 


__ 

+ 



K 


0 


^ = _k([^ 

dr [[ d h 


P{Di)+U^^)vec). 


(14a) 

(14b) 


Quasi-steady state analysis: letting (5 = 0 freezes D and 
k at the quasi-steady state on which Di = lj ,Vi £ IN. 

Hence, the reduced-system is 


dl , n dV i i nf \ \ i dP ( D i)i ^ 

“ k(l m + { l) + U ~dD~ ]vec) 


(15) 


dh 

Linearizing ( fl5l ) at the Nash equilibrium point 1* gives, 

dl 


dr 


= -kB{i- r), 


(16) 


where —ki? is Hurwitz by Assumption [3] and the Gershgorin 
Circle Theorem [48]. Hence, the equilibrium point 1* is lo¬ 
cally exponentially stable under (f]~5]> . i.e., there exist positive 
constants o, and o 2 such that (denote the trajectory of ( 1 1 5| i 

as lr (t)) 


,(r)- 1*|| <p ie -^ T ||l r ( 0 )- 1*||, 


(17) 


given that ||l r (0) — 1*|| is sufficiently small. 

Boundary-layer analysis: Since (D e (l), K e (l)) satisfies, 


-I-L -LU 

■ D e (l) ' 


' ATI ' 

U t L 0 

[ £ e (l) J 

+ 

0 


= 0, 


(18) 































































it can be derived that D e (l), K e (l) are linear functions of 1 
" —I — L -LU 
U t L 0 


as the matrix 
Let 


Then, 


is non-singular. 


dP' 


dr 


i 

n.|£-. 



D =D D e 

K = K — K e . 


—I — L -LU 
U t L 0 


D' + D e 

K + K S 
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Nl 
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-s 


dD e 


dn 


<91 n, 

eT T 


<91 


N 


f )\/. w 

( -k ([-^- + + E ^ 


3=1 


u dP(D' i + „ 

‘ dW + £j=i lj) U)) 
-I-L -LU 
U t L 0 

,.tT 


i- 

D' 

-1 

~/ 

K 


-S 


d D e 


dn 


d\ ^ 

eT T 


m 


( -k ([ ^+^ + E^ 

3=1 


(19) 


N 


. dP(D' + El 1 l j ) ] 

1 dM + zUh) U)) - 


( 20 ) 


Hence, in f-time scale 


dP 


dt f 

_ 

dU 


dt 



-6 


-I-L -LU 
U t L 0 

btT - 

(—k([ 


i- 

D' 

-1 

— / 

K 


ap e 


<9k. 


91 T 

eT I 


<91 


dL 


N 


+ P(D'i + Y^ l j) 

3=1 


+ k- 3 1 


d(D[ + Ef =1 lj) 

Letting <5 = 0 gives the boundary-layer model of (IT4l) as 


=))- 


( 21 ) 


dP 


dt f 

_ 

dn 


dt 



Since the matrix 


—I — L -LU 
U t L 0 

-I — L -LU 
U t L 0 



d' 


K 


( 22 ) 


is Hurwitz, the equi¬ 


librium point of the boundary-layer model D = 0, k =0 
is exponentially stable, uniformly in (f, 1). 

Therefore, by Theorem 11.4 in [42], it can be concluded 
that there exists a positive constant S* such that for all 0 < 
S < 6*, (1*, 1 l*,K e (l*)) is exponentially stable under 

<UID. 


□ 

From the analysis, it can be seen that all the states in © 
stay bounded and 1(f) produced by © converges to the Nash 
equilibrium under the given conditions. 

In this section, a Nash seeking strategy is proposed without 
requiring the uniqueness of the Nash equilibrium. In the fol¬ 
lowing section, energy consumption game for HVAC systems 
where the Nash equilibrium is unique will be considered. 


V. Energy Consumption Game among A Network 
of HVAC Systems 

For HVAC systems, the load curtailment cost may be 
modeled as [38] 

Vi(l i ) = v i &(li-ii) 2 ,ie IN, (23) 

where Vi and £ ?: are thermal coefficients, > 0 and l % is 
the energy needed to maintain the indoor temperature of the 
HVAC system. When the pricing function is 

N 

P(I)=aJ2h+Po, (24) 

2—1 

where a is a non-negative constant and a < miiiigiN for 
N > 3, the uniqueness of the Nash equilibrium is ensured 
[38], 

Based on this given model, a Nash seeking strategy 
will be proposed in this section for the players to search 
for the unique Nash equilibrium (by assuming that a < 

2iy . 

min ig ]N N _^ for N > 3 in the rest of the paper). 


A. Nash Equilibrium Seeking for Energy Consumption Game 
of HVAC Systems 

Lemma 1: The energy consumption game is a potential 
game with a potential function being 


N N N N 

q ( i) = E>*£M*) 2 +Z>( E (3 )^+Ek 2 +fo(z)- 

i— 1 i=l 2=1 

(25) 

Proof: Noticing that 


dQ{ 1 ) 
dU 


dCj{\) 

dh 


,V* € IN, 


(26) 


the conclusion can be derived by using the definition of 
potential game. □ 

Based on the primal-dual dynamics in [43], the Nash 
seeking strategy for player i is designed as 


if. = -A - ^2 (A 

jeNi 

hi = (A — Dj) 

je-Ni 


Dj) — y ' (rei — k j) + Nii 

jeNi 

(27a) 

(27b) 


• — (J v • 

h = + P ( D i) + al i - Vil + Va) (27c) 

Vii=rhaVa(ir n -h) (27d) 

f] i2 = fh i 2vAh-lT ax )H&^ (27e) 

















































where rriij = Srriij for all i g i,j £ { 1 , 2 }, are fixed 
positive parameters and r ( 0 ) > 0 ,i G 1N,j G {1,2}. 

By introducing the orthonormal matrix U as in Section 
IIV-BI the system in (l27l > can be rewritten as 



—J — L 
t/ T L 
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' Nl ' 



+ 



K 


0 


• — O V' 

U = + P ( D i) + al i ~ Vil + Va) 


Vn = mivu(ir m - h) 

Va = m i2 r) i2 (li - l™ ax ), i G IN 


(28a) 

(28b) 

(28c) 

(28d) 


and 

^ = 0. (29) 

Theorem 3: Suppose that Assumption!]] is satisfied. Then, 
there exists (3 G ICC such that for each pair of strictly positive 
real number (A,v), there exists S(A,v) > 0 , such that 


\\x(t)\\ <j9(||x(0)||,5f) + v, (30) 


for all t > 0, ||x(0)|| < A under ( 1281 ) given that , q ij (0) > 
0,* G IN, j G {1,2}. In @. X (t) = [(1(f) - 1*) T , (D(f) - 
1 Eti («(*) - « e (l*)) T , Mt) - V *) T ] T , and n, r]* 

are defined in the subsequent proof. 

Proof: Following the proof of Theorem [2] by using singular 
perturbation analysis, the reduced-system in r-time scale is 
given by 

dl N 

- U) + ayjj +p 0 + ali - r\ iX +r] i2 ) 

3 =1 

^ = m il n il (r n -k) 

^T L =m i2Va (li-ir x ),i£ IN. 

" T (31) 

According to Lemma [Q the energy consumption game is 
a potential game. Hence, the Nash seeking can be achieved 
by solving 


min Q(l) 

subject to l™ in <h< Vf ax , i G IN. 


(32) 


In the following, we show that (OTb can be used to solve 
the problem in (132} . 

Define the Lagrangian function as L(\. rj) = Q(l) + 
- U) + vAh - iT ax )) where V = 
[Vn,Vi2,V2i,V22,---,Va,Vi2r--,V ni,Vn2\ T G R 2 + n . 
The dual problem for the minimization problem in (l32l> can 
be formulated as 


max min L(l,ri). 
jj> o 1 


The Hessian matrix of Q(l) is 


H = 


A 2 cl 


2 i^ 2^2 ' 2 a 


(33) 


a 

a 


2vn£ 2 n + 2 a 


Since a < minjgjM -jfzfs f° r A > 3, H is positive definite by 
the Gershgorin Circle Theorem [48]. Hence, Q(l) is strictly 
convex in 1 as its Hessian matrix is positive definite. Noting 
that the inequality constraints are linear, the problem in (l32l) 
has strong duality [47]. Hence, 1* is the optimal solution to 
the problem in (l32t if and only if there exists q* G It 2 ' such 
that (1*, r]*) is the saddle point of L( 1, rf) by the saddle point 
theorem [47]. 

By defining the Lyapunov candidate function as [43] 


V L =i(l-l*) T k- 1 (l-l*)+ 


EE - (Vij - Vij - Vijlogiriij) + Vijlog(r]C)), 

i i • n ij 
2=1 7 = 1 J 

(34) 


where (1*, 77 *) is the saddle point of L{ 1, rf), it can be shown 
that the saddle point of L(l, 77 ) is globally asymptotically 
stable under (OTb by Corollary 2 of [43] given that ( 0 ) > 

0,f GIN,JG {1,2}. 

Hence, the strategy in ( OTb enables 1 to converge to the 
Nash equilibrium of the potential game asymptotically. 

Combining this result with the exponential stability of the 
boundary-layer system (see Theorem [2] for boundary-layer 
analysis), the result can be derived by using Lemma 1 in 
[44] (see also [45] for more details). □ 


B. Nash Equilibrium Seeking for Energy Consumption Game 
of HVAC Systems with A Unique Inner Nash Equilibrium 

In the following, energy consumptiongame with a unique 
inner Nash equilibrium is considered [j. If the constraints 
do not affect the value of the Nash equilibrium, the Nash 
seeking strategy can be designed as 
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By introducing the orthonormal matrix U as in Section 
IIV-BI the system in (l35b can be rewritten as 
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Theorem 4: Suppose that Assumption!]] is satisfied. Then, 
there exists a positive constant <5* such that for every 


1 To make it clear, in this paper, we say that the Nash equilibrium is 
an inner Nash equilibrium if the Nash equilibrium satisfies lV x ' L ' n < l* < 
I™ ax ,Vi G IN, i.e., the Nash equilibrium is achieved at 5^. — 0,Vz G IN. 



























0 < 5 < d*, (1(f), D(f), n(t)) converges exponentially to 
(r, 1 J2? =1 l*,K e { 1*)) under {36}. 

Proof: Following the proof of Theorem [2] by using singular 
perturbation analysis, the reduced-system at r-time scale is 
given by 
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From {38} . it can be derived that 

T = —kin, (39) 

where 1 (f) = 1 — 1 *. 

Hence, it can be shown that in r-time scale 

||T(r)|| < / max »gw{fc t } e - min igN {fc) } {H}r ||'l( Q )|| 

V rnin (C 

(40) 

by defining the Lyapunov candidate function as Vl = 
iFk-n [29], 

Combining this result with the exponential stability of the 
boundary-layer system (see Theorem U for boundary-layer 
analysis), the conclusion can be derived by Theorem 11.4 in 
[42] (see the proof of Theorem [2] for more details) . □ 

Remark 2: When the constraints do not affect the value 
of the Nash equilibrium, the updating strategy in {35} is a 
special case of the one in { 8 }. It can be seen that the result 
in Theorem @] is stronger than the result in Theorem [2] under 
the given HVAC model. 


C. Energy Consumption Game of HVAC Systems with Stub¬ 
born Players 

In this section, a special case where some players commit 
to the coordination process while keeping a constant energy 
consumption is considered. Without loss of generality, we 
suppose that player i is a stubborn player and updates its 
action according to 
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where ll is the constant energy consumption of player 
i. Furthermore, all the rational players adopt {35} if the 
constraints do not affect the value of all the players’ best 
response strategies, else, all the rational players adopt (ITT} . 

By introducing the orthonormal matrix U as in Section 
IIV-BI then 
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in which the ith component of 1 is fixed to be If, and 


= 0 . 


Furthermore, 
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if all the rational players adopt (l27l) . and 
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if all the rational players adopt {35} . 

Corollary 1: Suppose that Assumption |T] is satisfied. 
Then, there exists /3 G ICC such that for each pair of strictly 
positive real number (A,v), there exists S(A,v) > 0, such 
that 

||x hr (A)||</3(||A br (0)||,5f) + t;, (46) 


for all f > 0, ||x br (0)|| < A under E} and gl 
given that r] k j(0) > 0 ,k G IN, k f i,j G {1,2}. 
In {46}. X br (t) = [U(t) T ,(D(t) - l(Ef=i ,j^ l f + 

lh{t),--- Ji-1 (t),l i+1 (t),---,i N (t)] T , lj(t) = lj{t) - 
lj r , j G IN, j f i and iff denotes the best response strategy of 
player j, l_j is the concatenated vector of lj,i G IN, j f i, 
r)_ i , rjf) are defined in the subsequent proof. 

Furthermore, if the constraints lj l%n < lj < /"“Ej G 
IN, j f i do not affect the values of the best response 
strategies, then, there exists a positive constant S* such that 
for every 0 < S < S*, (I_i(f),D(f) - l(E^=i+ 
If), K(t) — /5 e (l br i , If)) converges exponentially to zero under 
© and {45}. 

Proof: Following the proof of Theorem [2] by using singular 
perturbation analysis to get the reduced-system for both 
cases. The first part of the Corollary can be derived by 
noticing that the following is satisfied, 
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Then, the dual problem is 

max mini/ (l_j. «_,•). 

r)^i >0 l-i 

Noticing that 1^ is the best response strategies of the rational 
players if and only if there exists r/ b f i £ R+ N ^ such that 
(} b -i, rf-i) is the saddle point of L (l_j, the rest of the 
proof follows that of Theorem [3] 

If the constraints < lj < £ IN, j ^ i do not 

affect the value of the best response strategies and all the 
rational players adopt (l35l >. then, the reduced-system at the 
quasi-steady state is 
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Writing (l50l) in the concatenated form gives 
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with hj = 2i/j£? + 2a. Since a < min, ;e iN for N > 3, 
it can be derived that the matrix Hi is symmetric and strictly 
diagonally dominant with all the diagonal elements positive. 
Hence, the matrix — kl/i is Hurwitz by the Gershgorin Circle 
Theorem [48], 

By similar analysis in Theorem Q] the conclusion can be 
derived. □ 

Remark 3: In this Corollary, only one player (i.e., player 
i) is supposed to be stubborn. However, this is not restrictive 
as similar results can be derived if multiple stubborn players 
exist. 

Remark 4: In the proposed Nash seeking strategy, the 
players only communicate with its neighbors on Di and /{,. 
They do not communicate on their own energy consumption 
li. Hence, the proposed Nash seeking strategy does not 
lead to privacy concern for the users. The work in [29] 
provided an extremum seeking method to seek for the Nash 
equilibrium in non-cooperative games. However, the method 
in [29] can’t be directly implemented for Nash seeking 
in the energy consumption game if the aggregate energy 
consumption is not directly available to the players. 



Fig. 2: Communication graph for the electricity users 


TABLE I: Parameters in the simulation. 



user 1 

user 2 

user 3 

user 4 

user 5 

I (kWh) 

50 

55 

60 

65 

70 

l™“(kWh) 

60 

66 

72 

78 

84 

kWh) 

40 

44 

46 

52 

56 

a 

0.04 

PO (MU/kWh) 
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VI. Numerical Examples 

A. Simulation Setup 

In this section, we consider a network of 5 commer¬ 
cial/industrial users that are equipped with HVAC systems. 
The electricity users communicate with each other via an 
undirected and connected graph as shown in Fig. [2] The cost 
function for electricity user i is 

CtiU, I) = Vig(li - Uf + P(l)h, (52) 

where the pricing function P(l) = aJ^iLih + Po [38]. 
Without loss of generality, suppose that for all i £ IN 
are normalized to 1 in the simulation. For i £ {1, 2, 3,4, 5}, 
l] nln = 0.8 k and l] nax = 1.2 h except that in Section [VLBl 
imm anc | jmax are set se p ara t e ly. The parameters are listed 
in Table |B 

B. Energy Consumption Control of HVAC Systems 

In this section, we suppose that If 171 = 45 and = 
55. By direct computation, it can be derived that the Nash 
equilibrium is 1* = (45,46.4, 51.3, 56.2,61.l)(kWh). The 
equilibrium aggregate is l* = 259.9(kWh). Hence, the Nash 
equilibrium is not an inner Nash equilibrium. 

The simulation results by using the seeking strategy in 
(l27l > are shown in Figs. [3]|4] 

Fig. [3] shows the users’ electricity energy consumptions 
produced by the proposed seeking strategy in (l27l) and Fig. 
Q] indicates that the users’ estimations on the aggregate 
energy consumptions converge to the actual aggregate energy 
consumption. 

From the simulation results, it can be seen that the energy 
consumptions produced by the proposed method converge to 
the Nash equilibrium of the energy consumption game. 

2 MU stands for Monetary Unit. 


























Fig. 3: The users’ energy consumptions produced by the 
proposed seeking strategy in d27l ). 



Fig. 4: The actual aggregate energy consumption and the 
users’ estimations on the aggregate energy consumption 
produced by the proposed seeking strategy in (ITH) . 


C. Energy Consumption Control of HVAC Systems with A 
Unique inner Nash equilibrium 

In this section, we consider the energy consumption 
game under the setting in Section IVI-AI By direct cal¬ 
culation, it can be derived that the Nash equilibrium is 
1* = (41.5, 46.4, 51.3, 56.2,61.l)(kWh). The equilibrium 
aggregate is l* = 256.7(kWh). The Nash equilibrium is an 
inner Nash equilibrium and the seeking strategy in (l35l > is 
used in the simulation. The simulation results produced by 
the seeking strategy in (l35l > are shown in Figs. |5][6] 



Fig. 5: The users’ energy consumptions produced b) 
proposed seeking strategy in (l35l >. 


From the simulation results, it can be seen that the 
users’ energy consumptions converge to the unique Nash 
equilibrium. 


D. Energy Consumption Control with Stubborn Players 

In this section, we suppose that player 5 is a stubborn 
player that commits to a constant energy consumption If = 
100(kWh). Then, player 1-4’s best response strategies are 
40.8(kWh), 45.7(kWh), 50.6(kWh), and 55.5(kWh), respec¬ 
tively. The aggregate energy consumption is 292.7(kWh). In 
the simulation, the rational players adopt (f35l) to update their 
actions. The stubborn player uses the seeking strategy in d4lt . 
The simulation results produced by the proposed method are 
shown in Figs. [7]|8] 

It can be seen that with the presence of the stubborn player, 
all the other players’ actions converge to the best response 
strategies with respect to the stubborn action. 

VII. Conclusions and Future Work 

This paper considers energy consumption control among 
a network of electricity users. The problem is solved by 
using an aggregative game on an undirected and connected 
graph. To estimate the aggregate energy consumption, which 
is supposed to be unknown to the players during the Nash 
seeking process, an average consensus protocol is employed. 
The convergence property is analytically studied via using 
singular perturbation and Lyapunov stability analysis. A 
general energy consumption game where multiple Nash equi¬ 
libria may exist is firstly considered. A Nash seeking strategy 
based on consensus is proposed to enable the users to search 
for the Nash equilibrium. Energy consumption control of 
HVAC systems with linear pricing functions is then studied. 
Convergence results are provided. Furthermore, stubborn 
players are investigated and it is shown that the rational 
players’ actions converge to the best response strategies. 

For future directions, the following aspects would be 
considered: 

1) The design of incentive provoking mechanisms. As 
Nash solution is usually not efficient from the system- 
level perspective, socially optimal solution might be 
preferred if coordination is allowed. Incentive provok¬ 
ing mechanisms can be designed to motivate the elec¬ 
tricity users to coordinate such that system efficiency 
can be improved [5], 

2) Analysis of the energy consumption game with the 
existence of cheaters. This includes the detection of 



Fig. 6: The actual aggregate energy consumption and the 
users’ estimations on the aggregate energy consumptions 
produced by the seeking strategy in (1351) . 
























































Fig. 7: The users’ energy consumptions produced by the 
proposed method (i.e., the rational players adopt (l35l > and 
the stubborn player adopts (ini '). 


cheaters, the design of penalty (e.g., [49]) or reward 
algorithms to prevent cheating behaviors, etc. 

3) Nash seeking for energy consumption game under 
various communication conditions. 
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